A Parallel Algorithm for Nonnegative Matrix Factorization Based on Newton Iteration

Nonnegative Matrix Factorization (NMF) is a technique to approximate a nonnegative matrix as a product of two smaller nonnegative matrices. The guaranteed nonnegativity of the factors allows interpreting the approximation as an additive combination of features, a distinctive property that other widely used matrix factorization methods do not have. Several advanced methods for computing this factorization exist, in this paper, we will focus on an algorithm based on Newton iteration. It was adapted for parallel execution on computer clusters, utilizing the message passing communication paradigm. Results of multiple experiments, which were run on a system with up to 80 processor cores, will be presented. We used images as input matrices, which makes it possible to get a visual impression of the NMF approximation quality. The measurements show that for sufficiently large workloads, the parallelized Newton iteration algorithm achieves an almost linear speedup, which makes it a promising candidate for large-scale NMF computations.